Modulated Vortex Lattice in High Fields and Gap Nodes 
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The mean field vortex phase diagram of a quasi two dimensional superconductor with a nodal 
d-wave pairing and with strong Pauli spin depairing is studied in the parallel field case in order 
to examine an effect of gap nodes on the stability of a Fulde-Ferrell-Larkin-Ovchinnikov (FFLO)- 
like vortex lattice. We find through a heuristic argument and a model calculation with a four-fold 
anisotropic Fermi surface that the FFLO-like state is relatively suppressed as the field approaches 
a nodal direction. When taking account of a couple of experimental results altogether, the present 
result strongly suggests that the pairing symmetry of CeCoIns should be of d^y-type. 
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In a recent paper (denoted as I hereafter), we 
have examined the vortex phase diagram of quasi two- 
dimensional (Q2D) type II superconductors with strong 
Pauli paramagnetic (spin) depairing by focusing on H || c 
case with a field H perpendicular to the superconducting 
layers. In contrast to earlier works taking account 

of both the orbital and spin depairing effects of the mag- 
netic field in clean limit, the orbital depairing has been 
incorporated fully and nonperturbatively there 0, and 
two new results opposite to those suggested previously 
0,0] were found. First of all, the mean field (MF) transi- 
tion at the Hc2 (T)-line changes from the familiar second- 
order one to a first-order (MF-FOT) 0, H, 111 one at a 
higher temperature T* than a region in which a Fulde- 
Ferrell-Larkin-Ovchinnikov (FFLO)-hke modulated 
vortex lattice may appear. This feature is consistent with 
data of CeCoIns in H || c H H i, 0. Second of all, a 
second order transition curve ffpFLO (T) between such a 
FFLO-like and ordinary vortex lattices remarkably de- 
creases upon cooling. Interestingly, these two results are 
also consistent with more recent data of CeCoIns, sug- 
gesting a structural transition to a FFLO state, in H _L c 
[3, IjOlllll H^ . A recent ultrasound measurement 
also shows that the suggested FFLO state is, as we have 
argued in I, a kind of vortex lattice. However, it should 
be further examined theoretically whether this qualita- 
tive agreement with the data in H _L c is justified or 
not. 

In this paper, results of an application of analysis in I 
to a model for the H _L c case are reported. By includ- 
ing the contributions, neglected in previous works 
from the nonCaussian (|A(r)|'* and |A(r)|^) terms of the 
Ginzburg-Landau (GL) free energy to the spatial gradi- 
ent parallel to H, where A(r) is the pair-field, we find 
that the relative position between T* and ifpFLO-line is 
qualitatively the same as in H || c case as far as a 
spin depairing strength realistic in bulk superconductors 
is used, and that, at least close to i/pFLOj the LO state 



with periodic nodal planes perpendicular to H of 
|A| is more stable than the FF state 0,0 composed of 
a phase-modulation with keeping |A| fixed. 

A special attention is paid in this paper to a notice- 
able in-plane angular dependence of the FFLO curve 
Hvv T,n(T) found in specific heat and magnetization 
data of CeCoIng: The observed FFLO curve in H \\ 
[110] lies at higher temperatures than that in 11 [100]. 
This i/pFLO-anisotropy is much more remakable [9| than 
that of Hc2 (T) and may give a decisive information on the 
four-fold anisotropy of the gap function. As far as an in- 
plane Fermi velocity anisotropy is negligible, it is heuris- 
tically predicted through the following simple argument 
how a gap anisotropy results in a -ffpFLO-a-nisotropy: 
Near the gap nodes where the superconducting gap Ak 
is small, the coherence length £k — Tivp/Ak defined lo- 
cally in the k-space is longer [13]. An orbital limiting 
field -fforb(O) is inversely proportional to the square of 
an averaged coherence length in the plane perpendicular 
to H and hence, is minimal when H is directed along 
the four-fold symmetric gap nodes (or minima). Since a 
higher Hoih will lead to a relatively stronger effect of spin 
depairing, the FFLO curve and T*, induced by the spin 
depairing, are expected to lie at higher temperatures as 
H is located along a gap maximum. If comparing the 
expected iJppLo-anisotropy with the observations 9, 1^ 
in CeCoIns, we inevitably reach the conclusion that, in 
agreement not with the original argument 4] favoring a 
(i2.2_j^2-pairing just like in high-Tc cuprates but with a re- 
cent report on low H specific heat data Q , a node (or a 
minimum) of the gap function of CeCoIns is located along 
the [100]-direction, implying a d^jy-pairing state. Below, 
we will show how this conclusion is reinforced through a 
microscopic derivation of -ffpFLo(2^) taking account of a 
possible in-plane four-fold anisotropy of the Fermi surface 
(FS). The present result might require a serious change in 
the picture on the pairing mechanism of CeCoIns based 
upon similarities on the normal state properties, includ- 
ing the presence of antiferromagnetic fluctuation, to the 
high Tc cuprates . 
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First, let us sketch the outhne of MF analysis for 
H II c. Throughout this paper, we assume H = Hx and 
the d-wave gap function — \/2cos(20) or V2sin(20), 
where (p is the azimuthal angle in the a-h plane. Within 
the lowest {N = 0) Landau level (LL), the GL free energy 
density in the MF approximation takes the form 



ao(Q)(|Ag)p) + ^(|AS)|^) 



^(|aS¥> 



C0+C2g'+C4Q^ (1) 



The essential part of the MF analysis in I is to derive 
the coefficients, ag, V4, Vg, C2, and C4 by starting from 
the weak-coupling BCS model with a Zeeman (Pauli- 
paramagnetic) term. Here, N{Q) is an averaged density 
of states (DOS) at the Fermi level, and ( ) is the spa- 
tial average on y and z. A(r) was expanded in terms 
of the LLs as A(r) = Ag^'(2/, z)wQ(a;), and the 

higher LLs were neglected above. For the LO (FF) state, 
uq{x) takes the form cofi{Qx) (exp(iQa;)). A Q2D FS 
with a circular form in y-z plane was assumed, although 
an in-plane anisotropy will be conveniently included as 0- 
dependences of the Fermi velocity and DOS (See eq.(5) 
below). For an example, a^iQ) is, after performing k- 
integrals and introducing a parameter integral, expressed 
by 



iV(0)ao(Q) = I^u*q{x)(^j^^^2t:T 



where ( )x denotes the spatial average on x, pqH is the 
Zeeman energy, and A^(0)|(7| is the dimensionless pairing 
interaction strength. The function g^'^\p^—idx) has the 
form 



so that the resulting Hyvlo{T) is a second order tran- 
sition line. If the effective strength of spin depairing 
^o-fforb/(27rfcB'7co) IS of Order unity or larger, a phase 
diagram derived numerically in this manner includes a 
^^FFLO (7')-line decreasing upon cooling, where /ig-ff is 
the Zeeman energy, and is the orbital limiting field 
in 2D limit. In Ref.l where the V4 and Vg-contributions 
to C2 were neglected, the LO and FF states had the 
same -ffpFLO-line, while we find that the instability of 
the straight vortex lattice into the LO vortex state [j, 1^ 
occurs at a slightly higher tempeature than that into the 
FF state 0,0 ■ Hence, at least close to iJFFLo(r), the LO 
state becomes the ground state in -ffFFLO < H < Hc2- 
Further, we find that the Ve-contribution to C2 is quan- 
titatively negligible, while the ifpFLO-line is pushed by 
the corresponding V4-contribution down to a lower tem- 
perature region in which iJc2 and the vortex state just 
below it are described by the iV = 1 LL. Thus, at least 
within the weak-coupling BCS model, a FFLO state in 
H II c rarely occurs because such a iV = 1 LL vortex 
lattice has no FFLO-like modulation X]. We guess that 
a slight specific heat anomaly |3] in CeCoIns in H || c at 
low enough temperatures may be rather due to a transi- 
tion between straight vortex lattices in = and A^ = 1 
LLs. A detailed study of this transition into a A^ = 1 LL 
state will be reported elsewhere. 

Now, let us turn to the H _L c case. Although, in 
principle, the above analysis can be extended to a Q2D 
system with a cylindrical FS under H perpendicular to 
the cylindrical axis, we have chosen to work in an ellip- 
tic FS elongated along z[\\ c)-axis and with the disper- 
sion relation = h"^ J2j=x,y,z^i^^'] / C^"^) under H || x 
in order to make numerical calculations more tractable, 
where Jx = Jy = 7""'^^^, and 7z = 7 with 7 > 1 and 
a constant m. We expect the case with a moderately 
large 7-value to qualitatively describe essential features 
in the realistic Q2D case. By isotropize the k vector 
as kj = jjkpfj, where f — (cos(/)sin0, sin0sin0, cos0) is 
the unit vector in the spherical coordinate, the veloc- 



ity V on FS is written as Vj = 7^ "^^FT^j- A Jacobian 

g'^^^Pj—idx) — N{0)exp{—p'^Vp/Ar'j^)cos{—ipvFdxl?) \/ jsin^O + ^~'^cos^9 accompanying the angular integral 

along FS is exactly cancelled by the angular dependence 

where rn is the magnetic length, and vp the Fermi ve- 
locity. The extension, oat, of oq to the A^-th LL is given 
by multiplying eq.(3) by £N{p'^v'j,/2rjj), if keeping just 
terms diagonal with respect to the LLs, where Cisi{x) 
the A^-th Laguerre polynomial. The coefficients Vi{Q) 
and V(,{Q) are derived in a similar manner to above. The 
coefficients C2 and C4 arise from the Q-dependences of oq, 
Vi, and Vq. 

The onset T* of MF-FOT at Hc2 is determined by 
V4(0) = irrespective of the details of higher order non- 
Gaussian terms of the GL free energy, while -ffFFLoC?^) 
is defined by C2 = under the condition C4 > 0. We 
have verified that the latter condition is always satis- 
fied throughout the computations in the present work. 



of DOS N(e) = N{Q)vf/ ^Y.jv'^r Again, the in-plane 
(four-fold) anisotropy of FS will be first neglected. Then, 
the GL free energy within A^ = LL takes the form of 
eq.(l), and the function g'^^\p, —'^dx) appearing in ao((3) 
(see eq.(3)) is replaced in the present case by 



(0)/ 



idx) = 



X coa{-ipVxdx), (4) 



where Vy^ — 777 



77? ^vl- 



The parameter fj is 
insensitive to the uniaxial anisotropy 7 but dependent 
on T and needs to be determined by maximizing Hc2(T). 
By focusing on the low T region, we find that fj takes 
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FIG. 1: H-T mean field phase diagram obtained using 7 = 
3 and witli no in-plane FS anisotropy. The transition or 
crossover positions in H || gap- maximum (|| gap-node or mini- 
mum) are expressed by the solid curves and filled circle (chain 
curves and open circle). The dotted curve and open triangle 
denote, respectively, the MF-FOT line and the position at 
which the two solid curves ajv(O) = in iV = and 1 merge 
with each other. 



a value between 0.4 and 0.5 depending on the relative 
angle between H and a nearest nodal direction. Using 
this parameter, the anisotropy in spatial variations of 
A(r) within the y-z plane is given by 7/77. Except for 
modifications indicated above, the corresponding quartic 
and 6th order terms of the GL free energy are derived 
by closely following the analysis in I. We choose a\\ — 

/^o-ffori~^^(^)/^B7cO as a measure of the spin depairing 
strength in H _L c, where hI^^^^\q) is the orbital limiting 
field in the isotropic case. 

In Fig.l, the resulting phase diagram is shown to illus- 
trate how the i?FFLo('7^)-position depend upon the rela- 
tive angle between H and the nodal directions. Thin solid 
(chain) curves are defined by aAr(O) = 0, and the Hc2{T) 
m. T > T* in each case is given by each ao(0) = line. 
In agreement with the heuristic argument given earlier, 
HpFhoiT) and T* are shifted to higher temperatures as 
the in-plane field is directed along a gap-maximum, re- 
flecting an enhanced spin depairing in this field configura- 
tion. As in H II c, the FFLO state at least close to iJpFLO 
has the LO-like variation. We have verified by combining 
our numerical calculations with an analytical calculation 
with the orbital depairing perturbatively included that 
such an in-plane iJFFLO-anisotropy is absent without the 
orbital depairing (i.e., when a|| = 00) and monotonously 
increases with decreasing a\\ . In contrast, it is not easy to 
properly predict the corresponding anisotropy (in-plane 
angular dependence) of Hc2{T)-cmve: First, the depres- 
sion of Hc2 due to the spin depairing is larger as the 
corresponding fforb(O) is higher, and hence, the Hc2- 
magnitude may not have a monotonous a||-dependence. 
Second, the MF-FOT line of Hc2 is directly determined 
by the details of the nonGaussian terms other than the 




FIG. 2: Results corresponding to Fig.l in the cases (a) (|/3| — 
0.2) and (b) (|/3| = 0.1) defined in the text. 



quartic one in GL free energy and hence, is quanti- 
tatively affected by our assumption of keeping the non- 
Gaussian terms only up to |A|^ in eq.(l). Actually, the 
rapid increase of MF-FOT line on cooling just below T* 
arises due to an extremely small V6(0) near T* and might 
flatten if we can numerically include the |A|* and higher 
order terms. In contrast, the Vg-contribution to C2 (i.e., 
to HpFi^o{T)) was negligible, like in H || c case, consis- 
tently with the smallness of Vi5(0) mentioned above. We 
expect that the -ffpFLO C?") curve is less sensitive to the 
neglect of the |A|^ and higher order GL terms. For these 
reasons, we will focus hereafter on T* and -//fflo which 
directly measure the (effective) spin depairing strength. 
The resulting anisotropics of T* and -ffpFLO in Fig. 1 qual- 
itatively agree with those of CeCoIns in H _L c UJ if 
a gap node (or minimum) is located along [100]. As al- 
ready mentioned, the MF-FOT line in = LL needs 
to lie above the corresponding ai(0) = line in order for 
HpFhoiT) to be realized as a transition line. As Fig.l 
shows, this condition manages to be satisfied in contrast 
to the H II c case. 

In order to examine how the result in Fig.l is affected 
by the in-plane FS anisotropy, let us next introduce it 
as a Fermi velocity anisotropy in a similar manner to 
Ref.El hke 



VF 



vf{(13) ^vf{1 + /3cos(40)), 



(5) 



where |/3| < 1, accompanied by the replacement A^(0) 
N{0)vf/vf{(I)) in any angular integral (see eq.(4)). Ex- 
cept these replacements in our calculation, the deriva- 
tion of phase diagrams is quite the same as that of Fig.l. 
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When /3 > (< 0), the Fermi velocity becomes maximal 
(minimal) along x. By combining these two cases with 
the two candidates, y/2cos{2(j)) and •\/2sin(20), of w^, we 
have four different cases of the relative anisotropics under 
a fixed H || x. We will classify them into two categories, 
(a) — \/2cos(2(/<) with /? < and = \/2sin(2(/)) 
with /3 > 0, and (b) = \/2cos(2(/)) with /3 > and 
W0 = •\/2sin(2^) with^_^< 0. This classification is mo- 
tivated by the result that, in the category (a), the 
Fermi velocity anisotropy and the pairing anisotropy fa- 
vor two different orientations, competing with each other, 
of the square vortex lattices to be realized in four-fold 
anisotropic d-wave superconductors in H || c, while such 
a competition does not occur in (b). In Fig. 2, the re- 
sulting phase diagrams for the categories (a) and (b) are 
given. In the case (a), the angular dependences of iJpFLO 
and T* are weakened by the FS anisotropy compared 
with those in Fig. I, while the opposite tendency is seen 
in the case (b). This result can be understood as fol- 
lows by noting that the orbital depairing strength local 
in the k-space is measured in the present case by Vy in 
eq.(4) (Note that, in 2D limit, v1 is absent there). By 
focusing on the case with H parallel to a gap node and 
noting in the integrand of eq.(4), one will notice 



that a nonzero 
tions of v'j,. 



tends to increase (decrease) contribu- 
on average, when /? < (/3 > 0). Thus, 
an enhanced orbital depairing in H parallel to a node 
of the case (b) additionally reduces TJpFLO so that the 
difference between the two cases in Fig. 2 follows. Bear- 
ing in mind the general character of this interpretation, 
we believe that the results in Fig. 2 are not qualitatively 
changed by a refinement of microscopic description. 

The above results commonly show a i/pFLO (2^)-liiie 
shifting to higher temperatures as the in-plane field ap- 
proaches a gap-maximum and, compared with the data 
in CeCoIns imply a dxy-stdXe as the pairing state 

of this material. Although one might consider a pos- 
sibility of (ia;2_y 2 -pairing based on the fact that an ex- 
tremely strong FS anisotropy in the case (a) may re- 
verse the anisotropics of T* and -ffpFLOj such a strong 
FS anisotropy of the case (a) should result 0| in a 
square vortex lattice with an orientation due to the 
FS anisotropy and hence, contradicts not only the spe- 
cific heat data [Ijj but the observed orientation [13] of 
H II c square vortex lattice. Therefore, an inclusion of 
FS anisotropy reinforces our conclusion favoring a dxy- 
pairing, although a moderate FS anisotropy competitive 
in H II c with the gap anisotropy (i.e., of the case (a)) is 
needed for quantitative understandings. 

In conclusion, the mean field phase diagram of a type 
II superconductor with strong Pauli paramagnetic de- 
pairing and with a four-fold symmetric d-wave pairing 
was qualitatively studied in the parallel field case. The 
region in which the FFLO vortex phase appears is en- 
larged when the in-plane field is directed along a gap- 
maximum. This result is reinforced by including in-plane 



FS anisotropics and strongly suggests a dxy pairing as the 
best candidate of the gap function of CeCoIns in spite of 
the electronic similarities 15j to that of high cuprates. 
A reinterpretation of thermal conductivity data by Izawa 
et al. Q can be seen in Ref . [l^ . The present theory 
should be applicable to examining the pairing state of 
other materials, such as an organic material 0, ll^ . 
showing a remarkable Pauli paramagnetic depairing. 
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